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1.  CNETS  Study  Report  2-76,  "Observations  on  the  Use  and 
Evaluation  of  ECII-LP  Simulators  for  Aviation  Training" 
is  promulgated  for  information. 

2.  The  conclusions  and  recommendations  contained  in  the 
report  are  those  of  the  writer  and  are  not  necessarily  those 
of  the  Chief  of  Naval  Education  and  Training  Support. 


3.  This  publication  has  been  reviewed  under  the  provisions 
of  SECNAVINST  5600.16  and  is  apnroved. 
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1.0  SUMMARY . An  evaluation  of  the  training  effectiveness  of 


ECII-LP  simulators  was  undertaken  using  the  participant- 
observer  method.  Comparisons  were  made  between  two  aviation 
courses — pilot  FAM  which  used  the  ECII-LPs  to  familiarize 
student  pilots  and  instructor  pilots  with  the  T-2C  aircraft, 
and  NFO  FAM  which  used  transparencies,  study  guides,  MTUs,  and 
chalk  boards  to  present  familiarization  training  to  NFO  students 
who  were  transitioning  to  the  TA-4  aircraft.  Interviews, 
student  questions,  and  a review  of  final  test  scores  showed 
that  the  two  courses  were  probably  equally  effective  in  meeting 
the  training  objectives  if  allowances  are  made  for  differences 
in  student  aptitudes  and  skills,  previous  aviation  experience, 
and  length  of  the  training  period.  The  observations  found, 
however,  that  the  courses  also  provided  training  in  basic 
cockpit  procedures  although  the  final  tests  did  not  emphasize 
these  objectives  nor  were  the  ECII-LPs  being  used  effectively 
for  this  type  of  training.  Although  the  ECII-LPs  were  highly 
mobile  and  presented  training  information  in  a well  organized 
fashion,  the  rear  projection  visual  system  of  the  simulator 
was  found  to  be  inadequate,  and  panel  replacement  was  difficult. 
Instructors  were  not  provided  with  sufficient  training  or  support 
in  the  use  and  maintenance  of  the  simulators,  nor  were  necessary 
replacement  parts  readily  available.  The  reinforcing  and  problem 
solving  features  of  the  simulators  were  seldom  used,  nor  were  stu 
dents  allowed  to  operate  the  simulators  alone  or  in  small 
groups.  Recommendations  are  made  that  (a)  the  basic  cockpit 
procedures  training  objectives  of  the  FAM  courses  be  recognized 


officially,  (b)  completely  documented  training  scenarios 
which  relate  simulator  use  to  specific  behavioral  objectives 
be  developed,  (c)  better  maintenance  support — 0-level 
training  and  parts — be  provided,  (d)  improvements  be  made 
in  the  rear  projection  visual  system  and  panel  replacement, 
and  (e)  IMI  (Instructor  Managed  Instruction)  procedures  be 
adopted  in  order  to  avoid  problems  associated  with  differences 
in  student  aptitudes,  skills,  and  experience,  as  well  as 
variations  in  instructor  delivery  and  organization.  Implementa- 
tion of  these  recommendations  should  result  in  better  use  of 
CPTs  (Cockpit  Procedures  Trainers)  and  improved  aircraft 
performance.  A summative  evaluation  of  the  ECII-LPs  should 
be  delayed  until  major  course  modifications  have  been  made  and 
objective  performance  measures  of  training  effectiveness  have 
been  developed. 

2.0  ACKNOWLEDGEMENTS . This  report  was  most  highly  dependent 
on  the  cooperation  of  ADCS  K.  I.  Brant  (Chief  Petty  Officer-In- 
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help  provided  by  Mr.  Bob  Carter,  Education  Specialist  assigned 
to  NAMTRAGRU,  Millington,  Tennessee  is  also  appreciated.  The 
assistance  of  Mr.  Pete  Harris  in  reviewing  the  report,  and  of 
Miss  Teresa  Hindman  in  typing  the  report,  are  gratefully 
acknowledged. 


2 


3.0  PURPOSE.  The  information  and  recommendations  presented 


in  this  report  are  essentially  formative  in  that  the  purpose 
of  this  report  is  (a)  to  recommend  techniques  and  procedures 
for  more  effective  use  of  ECII-LP  simulators  in  aviation 
familiarization  (FAM)  training  prior  to  a formal  summative 
evaluation  of  the  training  effectiveness  of  ECII-LP  simulators, 
and  (b)  to  develop  and  recommend  a design  or  plan  for  a summa- 
tive evaluation  of  the  training  and  performance  effectiveness 
of  the  ECII-LP  simulators.  This  report  was  written  in  partial 
fulfillment  of  a joint  CNATRA/CNET  SUPPORT  plan  to  evaluate 
the  training  effectiveness  of  simulators  which  are  used  in 
several  Navy  undergraduate  aviation  training  programs,  including 
those  aviation  programs  which  use  ECII-LP  simulators  in  familiar- 
ization training.  This  joint  CNATRA/CNET  SUPPORT  plan  was 
approved  in  CNET  Second  Endorsement  Code  N-421  of  12  August 
1975  on  CNET  SUPPORT  letter  Code  N-2131  1500  of  26  June  1975. 

This  report  was  written  in  fulfillment  of  CNETS  Field  Task 
Assignment  No.  60082  which  was  assigned  to  manage  this  phase 
of  the  evaluation.  The  recommendations  made  in  this  report 
should  assist  instructors  and  progreun  managers  within  and  out- 
side the  Navy  Education  and  Training  Command  in  using  and 
procuring  this  and  similar  training  equipment  in  the  most  cost- 
and training-effective  manner  possible. 

4.0  BACKGROUND . Previous  research  in  a wide  variety  of  main- 
tenance training  situations  has  demonstrated  that  the  training 
effectiveness  of  FCII  simulators  is  equal  to,  or  better  than, 
other  training  equipment.  Other  training  equipment  usually 
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consists  of  Maintenance  Training  Units  (MTUs)  which  are  sub- 
systems or  components  of  standard,  operational  equipment 
(references  1,  2,  3,  and  4).  The  training  criteria  used  in 
this  previous  research  include  (a)  the  speed  and  accuracy  of 
normal  checklist  procedures  and  troubleshooting  performance 
(references  1 and  2) , (b)  the  speed  and  accuracy  with  which 

maintenance  manuals  are  used  and  normal  equipment  parts  are 
identified  and  located  (reference  3),  (c)  written  post-test 

scores  (reference  3) , and  (d)  attitudes  toward  instruction  and 
training  equipment  (references  1,  2,  and  4).  These  findings 
have  also  demonstrated  that  the  ECU  simulators  are  more 
cost-effective  than  other  training  equipment  (references  2 and 
3)  . 

The  ECU  simulator  is  manufactured  in  the  following  two 
models;  the  ECU  which  is  contained  in  a compact,  desk  console 
for  individualized  training,  and  the  ECII-LP  which  consists  of 
a large,  mobile,  mainframe  for  group  demonstrations  and  lec- 
tures. (Most  of  the  research  cited  above  has  analyzed  the 
training  effectiveness  of  the  ECII-LP  models.)  As  shown  in 
Appendix  A,  both  models  have  removable  display  (student)  panels, 
and  a visual  system  consisting  of  a random  access  projector  and 
a rear  projection  screen.  The  rear  projection  visual  system 
can  present  100  slides  for  each  training  scenario.  Both  models 
operate  from  similar  minicomputer  and  software  logic  systems. 
Although  the  two  dimensional  configuration  of  the  display  panels 
limits  structural  fidelity  for  the  Navy  aircraft  systems  which 
are  simulated,  the  functional  fidelity  of  the  models  is  extremely 
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high  (reference  4).  This  high  level  of  functional  fidelity  appears 
to  be  accounting  for  the  training  effectiveness  of  these  simula- 
tors in  the  research  previously  cited.  This  research  has 
centered  around  0-level  maintenance  training,  which  consists 
largely  of  knowledge  of  cognitive  factors — parts  identification 
and  comprehension  of  the  functional  relationships  which  exist 
among  the  components  of  an  equipment  system.  The  current 
ECUs  appear  to  be  especially  suitable  for  this  type  of  training. 
Although  the  low  level  of  structural  fidelity  probably  makes 
the  ECUs  unsuitable  for  advanced  training  of  complex  psychomotor 
procedures,  this  deficiency  may  be  overcome  in  the  next  genera- 
tion of  EC  simulators — the  ECIIIs — which  will  have  full  scale, 
three  dimensional  ("wrap  around")  consoles  and  mainframes. 

Underlying  the  functional  fidelity  and  training  effective- 
ness of  the  ECII-LPs  is  the  minicomputer  and  associated  software. 
These  characteristics  provide  the  ECII-LPs  with  programming  and 
reliability  advantages  which  are  not  found  in  MTUs.  Programming 
permits  the  simulator  to  be  interactive,  as  well  as  asily 
updated  as  functional  modifications  occur  in  associated  opera- 
tional equipment.  The  electronics  contained  in  the  minicomputer 
make  the  simulator  highly  reliable  and  maintainable. 

The  high  functional  fidelity  and  interactive  characteristics 
of  the  simulators  may  not,  however,  be  used  extensively  in  most 
of  the  Navy  aviation  maintenance  programs  for  which  the  simulators 
have  been  procured.  This  conclusion  was  made  from  the  training 
situations  analyzed  in  references  3 and  4.  As  mentioned  above. 
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the  performance  criteria  used  in  these  evaluations  was  limited 
largely  to  parts  location  on  operational  equipment.  This 
limitation  was  necessary  not  only  because  dedicated  aircraft 
could  not  be  spared,  but  also  because  the  students  were  not 
prepared  to  troubleshoot  test  problems,  or  even  describe  the 
procedures  involved  in  troubleshooting  the  test  problems. 

This  performance  effect  indicates  that  much  of  the  maintenance 
training  program  had  involved  rote  learning  and  memorization, 
and  that  functional  relationships  among  systems  and  system 
components  were  not  well  understood  by  the  students.  The 
functional  fidelity  and  interactive  characteristics  of  the 
ECII-LP  apparently  were  not  being  used  as  an  integral  part  of 
the  training  program.  Inasmuch  as  these  characteristics  account 
for  much  of  the  costs  associated  with  ECII-LP  simulators,  the 
failure  to  make  appropriate  use  of  these  characteristics  reduces 
significantly  the  cost-effectiveness  of  the  simulators.  In 
fairness  to  the  course  instructors,  however,  they  were  not 
provided  with  adequate  instructional  support,  either  in  the 
form  of  training  scenarios  or  simulator  training  methodology. 
They  appeared  therefore  to  use  the  ECII-LPs  in  much  the  same 
way  as  they  liad  used  the  MTUs  previously--as  training  aids  for 
demonstrating  parts  location  and  overall  (normal)  systems 
operation. 

In  recognition  of  the  above  limitations  and  advantages  of 
ECII-LPs  in  maintenance  training,  the  following  observations 
were  made  of  aviation  familiarization  courses — one  which  used 
ECII-LPs  and  the  other  which  used  more  conventional  training 
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aids  (MTUs,  study  guides,  and  transparencies).  The  participant- 


observer  method  was  used  in  making  these  observations--i .e. , 
the  author  participated  as  a student  in  the  two  courses,  being 
exposed  to  the  same  training  situations  as  any  other  student  and 
taking  the  required  tests.  The  observations  emphasize  the  func- 
tional descriptions  of  aircraft  systems  which  were  made  in  these 
courses,  and  the  techniques  used  to  present  these  descriptions. 
Recommendations  for  evaluating  the  training  effectiveness  of 
these  techniques  will  also  be  made. 

5.0  METHODS . The  following  section  contains  a description 

of  the  courses  which  were  observed,  and  the  interviews  made,  in 
conjunction  with  this  analysis. 

5.1  Course  Descriptions.  Two  courses  were  observed — the  pilot 
familiarization  course  for  the  T-2C  aircraft  {referred  to  below 
as  the  Pilot  FAM  course) , and  the  NFO  (Naval  Flight  Officer) 
familiarization  course  for  the  TA-4  aircraft  (referred  to  below 
as  the  NFO  FAM  course).  Pilot  FAM  used  ECII-LP  simulators, 
while  NFO  FAM  did  not.  Both  courses  have  been  offered  for  a 
number  of  years,  and  the  ECII-LPs  had  been  used  in  the  pilot 
FAM  course  for  over  a year  prior  to  the  present  evaluation.  In 
addition,  both  courses  were  taught  by  the  same  three  instructors. 

The  pilot  FAM  course  lasted  about  18  hours  (from  0800 
Wednesday  to  1100  Friday) , including  several  hours  for  review  and 
an  hour  for  testing.  The  total  enrollment  was  six  (including 
this  author) , and  the  aviation  background  of  the  student  group 
was  varied  (consisting  of  Navy  and  Marine  undergraduate  student 
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pilots,  and  prospective  T-2C  instructor  pilots).  (This  class 
size  and  composition  apparently  is  typical  of  this  pilot  FAM 
course.)  Although  the  purpose  or  specific  behavioral  objec- 
tives of  the  course  were  never  presented  (similarly  with  the 
NFO  FAM  course) , the  assumption  is  made  that  the  course  empha- 
sized Icnowledge  factors,  especially  the  location  of  indicators 
and  control  Icnobs,  handles,  and  switches  (referred  to  below  as 
displays  and  controls) , and  the  conditions  (normal  and 
emergency)  under  which  these  displays  and  controls  would  become 
operative  or  should  be  operated  by  the  pilot.  Display  and 
control  information  was  divided  according  to  the  eiqht  major 
aircraft  systems--electrical  (E) , instruments  (I),  canopy/ 
ejection  (CE) , oxygen/life  support  (OLS) , air  conditioning/ 
pressurization  (ACP) , power  plant/fuel  (PPF)  , hydraulics/ 
flight  control  (HEC) , and  communications,  navigation,  and  iden- 
tification (CNI) . For  each  system  this  information  was  further 
subdivided  into  forward  cockpit  (instructor  pilot)  and  aft 
cockpit  (student  pilot).  (In  NFO  courses,  the  NFO  student 
would  be  located  in  the  aft  cockpit.)  This  subdivision  was 
made  in  order  to  inform  the  student  about  transfer  of  system 
control  between  cockpits,  especially  the  conditions  under  which 
transfer  could  occur  and  the  conditions  (emergency  and  training) 
under  which  one  cockpit  (usually  the  forward  cockpit)  would 
retain  or  override  control  from  the  other  cockpit. 

For  the  pilot  FAM  course,  the  ECII-LPs  were  used  as  the 
primary  instructional  device  for  every  system  except  CE  and  OLS. 
Although  an  ECII-LP  was  available  and  used  for  OLS,  much  of  the 
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information  for  this  system  was  presented  by  film  and 
transr arencies . An  ejection  seat  had  been  removed  from  a 
T-2C  aircraft  for  use  as  an  MTU  in  CE  instruction.  A study 
guide  and  transparencies  also  supplemented  instruction  for  the 
eight  aircraft  systems.  The  study  quide  and  lectures  provided 
most  of  the  information  about  critical  (normal  and  emergency) 
conditions  and  parameters  under  which  the  various  systems 
would  operate  or  should  be  operated  (altitudes,  speeds, 
pressures,  weights,  temperatures,  and  so  forth).  A workbook 
which  consisted  of  completion  and  matching  questions  was  also 
provided  to  each  student.  These  questions  emphasized  the  most 
important  information  in  the  course,  and  also  contained  most 
of  the  40  test  items  on  the  final  test  (as  stated  by  the  instruc- 
tors at  the  start  of  each  instructional  unit) . 

The  NFO  FAM  course  was  similar  to  the  pilot  FAM  course, 
except  that  (a)  the  subject  aircraft  was  the  TA-4  (a  more 
advanced,  complex  jet  aircraft  than  the  T-2C) , (b)  the  students 

were  NFOs  and  not  pilots,  and  (c)  ECII-LPs  were  not  available. 

The  student  group  was  larger  (about  30)  and  less  varied  (only 
Navy  and  Marine  undergraduate  NFOs)  than  was  the  T-2C  student 
group.  NFO  FAM  lasted  about  11  hours  (from  0800  Thursday  until 
1100  Friday) , and  included  information  about  the  same  eight 
aircraft  systems  which  were  covered  in  the  T-2C  course.  Less 
information  was  provided  about  the  forward  cockpit  and  about 
direct  aircraft  control  in  the  NFO  FAM  course  than  in  the  pilot 
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FAM  course  because  the  NFO  students  are  only  transitioning 
through  the  TA-4  aircraft  prior  to  more  intensive  flight 
instruction  in  more  advanced  aircraft.  Inasmuch  as  the  NFO 
student  is  not  required  to  operate  this  aircraft  in  flight, 
the  major  purpose  of  the  TA-4  NFO  student  instruction  is  to 
introduce  or  familiarize  the  student  with  more  advanced,  high 
performance,  jet  aircraft  and  the  normal  and  emergency  condi- 
tions (and  associated  display  and  control  information)  under 
which  these  aircraft  operate.  This  information  will  permit  the 
NFO  student  to  assist  the  pilot  in  monitoring  critical  display 
information  during  flight,  warn  the  pilot  if  emergency  para- 
meters and  conditions  are  indicated,  and  take  appropriate 
corrective  action  if  the  situation  warrants.  Inasmuch  as 
transfer  of  TA-4  aircraft  control  to  an  NFO  student  would 
usually  be  limited  to  emergency  conditions,  emphasis  in  the 
NFO  FAM  course  was  placed  on  those  systems  most  likely  to  be 
monitored  and  operated  under  such  conditions — I,  CE,  OLS, 
and  CNI.  (CNI  is  also  important  to  the  normal  aviation  tasks 
performed  by  NFOs.)  Transparencies  and  student  guides  were  the 
primary  training  aids  used  in  the  NFO  course.  These  training 
aids  supplemented  the  critical  conditions  and  parameter  informa- 
tion which  was  provided  by  lectures  in  much  the  same  fashion 
as  the  ECII-LP  did  in  the  pilot  FAM  course.  The  NFO  FAM  course 
also  contained  a workbook  of  completion  and  matching  questions 
which  was  used  by  the  students  to  prepare  for  the  50-item 
final  test. 
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The  typical  instructional  unit  in  both  FAM  courses  was 
centered  around  one  of  the  eight  aircraft  systems,  was  delivered 
by  a single  instructor,  and  lasted  from  one  to  three  hours  (with 
a 10-minute  break  every  50  minutes) . The  unit  began  by 
having  the  instructor  explain  the  purpose  of  the  system  and 
the  location  and  operation  of  important  system  components 
(which  was  demonstrated  on  the  available  training  aids)  . 

During  these  demonstrations,  the  instructor  would  either  write 
critical  operating  conditions  and  parameters  on  a chalk  board 
or  refer  to  the  study  guides  for  this  information.  Normal 
system  operation  was  usually  explained  first,  followed  by 
operation  under  emergency  conditions.  (Emergency  conditions 
which  resulted  from  the  failure  of  components  within  the 
subject  system  or  failures  within  a support  system.)  Occa- 
sionally, after  covering  this  information,  an  instructor  would 
progress  through  a scenario  involving  several  of  the  emergency 
conditions  previously  presented.  The  instructional  unit  ended 
by  having  the  instructor  review  questions  from  the  workbook 
(the  answers  to  about  half  of  these  workbook  questions  were 
solicited  from  the  class  as  a whole) . Both  FAM  courses  could 
be  briefly  described  as  rapidly  paced,  competitive,  and  highly 
demanding  (the  student  would  probably  have  to  spend  at  least 
two  or  three  hours  a night  after  class  memorizing  and  organizing 
the  information  sufficiently  well  to  earn  over  90%  on  the  final 
test — even  with  the  information  provided  in  the  workbooks) . 
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5.2  Observation  Procedures. 


As  mentioned  earlier,  this  report 


is  essentially  a formative  review  of  the  FAM  courses,  and  is 
based  largely  on  observation  of  training  (instructor)  technique-, 
classroom  (student)  responses  to  these  training  techniques,  as 
well  as  interviews  with  instructors  and  students.  Quantitative 
or  statistical  data  were  not  collected  or  used.  Recommendations 
will  be  made,  however,  about  the  quantitative  data  which  may  be 
most  reliable  and  valid  if  future  summative  evaluations  are 
underta)cen.  A summative  evaluation  would  be  in  Iceepinq  with 
the  purpose  of  these  observations  and  recommendations,  which 
are  presented  not  only  to  modify  the  training  programs,  but  to 
recommend  modifications  which  can  be  tested  for  validity  using 
more  appropriate  analytical  and  statistical  procedures. 

6.0  RESULTS  AND  DISCUSSION.  The  following  sections  describe 
the  observational  and  interview  findings,  and  the  interpreta- 
tions of  these  findings. 

6.1  Instructor  Findings.  Observation  of  instructor  techniques 
in  both  FAM  courses  indicated  that  the  ECII-LP  made  the  pre- 
sentation of  training  information  better  organized  and  more 
convenient  than  MTUs,  transparencies,  or  the  chalk  board. 

These  advantages  could  be  most  readily  demonstrated  by  observing 
the  use  of  the  ejection  seat  MTU  in  the  CE  unit  of  both  courses. 
This  device  was  heavy  and  awkward  to  move,  difficult  for 
students  to  observe  (observation  was  limited  to  about  half  a 
dozen  students) , and  poorly  configured  for  instruction  (critical 
components  were  not  labelled  or  well  marked  and  could  not  be 
activated) . (An  ECII-LP  or  some  other  training  aid  should  be 
made  available  for  the  CE  instructional  unit.)  The  instructors 
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could  make  easy  and  direct  reference  to  system  components  on 
the  ECII-LPs,  and  ECII-LP  panel  graphics  had  excellent  two 
dimensional  fidelity  and  were  highly  visible.  The  trans- 
parencies which  were  used,  although  accurate  in  outline  form, 
had  low  structural  fidelity  and  could  not  be  seen  as  well  as 
the  ECII-LP  panels  because  of  poor  use  of  color  and  because 
projection  lighting  was  too  intense.  The  transparencies  were 
also  more  difficult  for  the  instructors  to  handle--they  often 
searched  through  several  transparencies  before  finding  the 
correct  one,  and  then  occasionally  reversed  the  projection. 

As  mentioned  previously,  the  ECII-LPs  had  the  added  advantage 
of  showing  the  functional  flow  or  relationships  between  systems 
and  system  components.  This  feature  permitted  the  instructor 
to  deactivate  a system  component  (or  support  system)  and  readily 
demonstrate  the  operational  effects,  and  then  demonstrate  which 
auxiliary  systems  or  components  would  correct  the  faulty  condi- 
tion. At  least  two  shortcomings  were  noted  in  this  feature, 
however.  First,  the  direction  of  the  functional  flow  or  rela- 
tionship was  not  evident  (this  information  had  to  be  provided 
by  the  instructor) . Secondly,  the  same  color  of  lights  was 
often  used  to  designate  functional  relationships  between  primary 
and  auxiliary  components  and  systems,  making  differentiation 
between  these  components  end  systems  difficult.  The  usefulness 
and  necessity  of  this  feature  in  the  present  training  context 
may  be  questionnable  as  well.  Much  of  this  functional  informa- 
tion could  be  presented  on  a set  of  carefully  prepared  slides 
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which  show  the  sequential  deactivation  and  activation  of  primary 
and  auxiliary  systems  and  components.  In  addition,  the  FAM 
instructors,  similar  to  the  aviation  maintenance  instructors 
who  were  referred  to  previously,  seldom  used  the  ECII-LPs 
for  the  purpose  of  demonstrating  functional  relationships 
between  primary  and  auxiliary  systems.  Although  they  (the 
FAM  instructors)  spent  a significant  portion  of  each  unit 
describing  emergency  conditions  and  procedures,  little  was 
done  to  demonstrate  this  information  on  the  ECII-LPs.  (They 
used  the  chalk  board  more  often  for  these  demonstrations.) 

Those  ECII-LP  demonstrations  which  were  performed  were 
occasionally  ineffective,  usually  because  incorrect  program 
inputs  (scenario  conditions)  were  entered  into  the  ECII-LP 
minicomputer  or  (more  rarely)  because  of  some  unreliability 
in  the  computer  electronics. 

Several  other  characteristics  were  noted  in  the  ECII-LP 
which  may  distract  from  the  use  or  training  effectiveness  of 
the  device.  Despite  the  mobility  of  the  device,  the  panels 
required  at  least  two  instructors  for  replacement.  This 
procedure  occasionally  took  10  to  15  minutes  while  another 
instructor  was  being  located,  and  replacement  did  not  always 
coincide  with  a regularly  scheduled  10-minute  break.  Another 
shortcoming  was  found  with  the  random  access  rear  projection 
visual  system.  Many  of  the  programming  difficulties  mentioned 
above  appeared  to  be  related  to  this  system,  which  was  used  to 
display  (a)  critical  parameter  information  associated  with  the 
training  scenario  being  programmed  on  the  panel,  or  (b) 
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interactive  information  such  as  whether  or  not  the  faulty 
condition  had  been  corrected.  The  difficulties  involved 
miscuinq  between  the  rear  projection  visual  system  and  the 
scenarios  which  had  been  programmed  on  the  ECII-LP  panels. 

(For  example,  while  demonstrating  ignition  of  the  second  engine 
on  the  T-2C  aircraft  during  a deck  battery  start,  the  FCII-LP 
panel  would  show  the  first  engine  at  65  percent  D.C.  power 
followed  by  ignition  of  the  second  engine,  while  the  informa- 
tion on  the  rear  projection  visual  system  read  "bring  first 
engine  to  65  percent  D.C.  power.")  Miscuing  appeared  to  be 
especially  disturbing  to  the  instructors,  perhaps  because 
they  were  afraid  that  they  were  perceived  by  the  students  as 
incompetent  to  operate  the  device,  or  unknowledgeable  about 
the  material. 

These  miscuing  effects  probably  were  involved  in  the 
instructors  not  using  the  device  more  to  demonstrate  functional 
flow  and  emergency  procedures.  Most  of  the  training  scenarios 
could  be  presented  on  the  device  without  this  visual  support 
and  still  be  of  some  training  benefit.  Worthwhile  interactive 
features  (if  operational)  would  be  lost,  however.  In  addition 
to  these  programming  or  miscuing  difficulties,  many  of  the 
rear  projection  visuals  could  not  be  seen  adequately  (even 
by  students  sitting  in  the  second  row  within  12  to  15  feet 
of  the  device) . Most  of  the  poor  visuals  included  photographs 
of  the  aircraft  (such  as  doors,  bays,  receptacles,  guages, 
and  so  forth) . These  visual  problems  were  most  likely  related 
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to  (a)  poor  photography  and  photographic  development 
(especially  inadequate  contrast  and  perspective  between 
subject  and  field) , (b)  poor  resolution  of  the  visuals 

through  the  projection  system  and  on  the  screen,  (c)  the  small 
size  (12  X 12  inches)  of  the  rear  projection  screen  (which 
should  be  at  least  double  this  size) , and  (d)  poor  lightinq 
in  the  classroom.  Much  of  the  information  provided  bv  the 
visuals  was  also  duplicated  on  the  panels,  in  the  study  guides, 
or  by  the  instructor.  Except  for  those  visuals  which  presented 
interactive  information  (which,  as  mentioned  above,  was  used 
only  rarely  in  the  present  course) , the  visual  system  could 
best  be  described  as  a redundant  and  unnecessary  appendage 
to  the  ECII-LPs. 

Interviews  with  the  instructors  indicated  that  they 
generally  were  pleased  with  the  ECII-LPs,  largely  because  the 
device  (and  instructional  delivery)  were  more  compact, 
organized,  and  convenient  than  other  training  aids.  They 
ac)cnowledged  that  they  device  was  extremely  useful  in 
demonstrating  functional  relationships  among  systems  and  in 
training  emergency  procedures.  Several  instructors  stated 
that  the  device  appeared  to  improve  final  test  performance 
by  an  average  of  five  to  ten  percentage  points.  Although 
accurate  comparison  data  were  not  available  to  substantiate 
these  opinions,  a review  of  the  final  grades  for  both  FAM 
courses  showed  performance  to  be  uniformly  high,  with  few 
failures.  The  student  pilots  did  better  on  the  average  than 
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the  student  NFOs,  but  this  difference  may  have  been  related 
to  variations  in  length  of  training,  aptitudes,  and  testing 
procedures.  The  instructors  did  not  complain  about  the 
programming  problems  previously  mentioned,  but  several  of 
the  instructors  stated  that  the  rear  projection  visuals  were 
inadequate.  Although  they  were  satisfied  with  the  overall 
reliability  of  the  ECII-LPs,  they  did  complain  that  maintenance 
support  was  a major  problem.  Troubleshooting  schematics  were 
not  available  for  the  minicomputer,  visual  system,  or  panels, 
nor  could  spare  parts  be  obtained  locally.  Troubleshooting 
and  minor  repairs  were  therefore  dependent  on  a few  experienced 
instructors,  a procedure  which  is  inefficient  because  of  the 
lengthy  and  spurious  OJT  involved,  as  well  as  interference 
with  regular  instructor  duties.  Such  a maintenance  procedure 
can  be  easily  jeopardized  if  these  experienced  instructors 
are  unavailable.  Procurement  of  major  parts  and  servicing 
involved  the  usual  requisitioning  procedures,  with  consequent 
delays  ranging  from  several  weeks  to  months.  These  short- 
comings seriously  degrade  the  maintenance  advantages  of  the 
device . 

Although  not  mentioned  by  the  instructors,  the  above 
observations  on  use  of  the  ECII-LPs  indicate  that  the  absence 
of  support  extended  as  well  to  instructional  delivery.  The 
ECII-LPs  apparently  were  sent  to  the  NAMTRADETs  with  minimal 
instructions  or  directions  for  using  the  panels  effectively 
for  training.  Instructions  and  directions  appeared  to  be 
limited  to  operation  of  the  programming  features.  Little 
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was  done  to  provide  the  instructors  with  complete  training 
scenarios  or  directions  for  the  use  of  the  functional  flow, 
emergency  procedures,  or  interactive  features  of  the  device, 
or  ways  in  which  the  device  could  be  used  for  individualized 
instruction  or  in  peer  group  training  situations.  Again, 
this  absence  of  instructional  delivery  support  may  account 
at  least  partially  for  the  minimal  use  of  the  ECII-LPs  in 
demonstrating  functional  flow  characteristics  and  emergency 
procedures  and  conditions. 

In  order  to  improve  the  maintenance  and  training  effec- 
tiveness of  the  ECII-LPs,  the  Navy  should  procure  the 
schematics  and  parts  necessary  for  local  0-level  maintenance, 
as  well  as  a set  of  detailed  and  organized  scenarios  for 
use  by  instructors  in  demonstrating  normal  and  emergency 
aircraft  system  operation.  In  the  current  training  situation, 
these  scenarios  should  cover  complete  instructional  units, 
and  should  integrate  the  ECII-LPs  with  other  existing  training 
aids  (especially  transparencies,  films,  and  study  guides). 

If  the  current  training  situation  is  modified  as  recommended 
below  (see  section  6.3),  these  scenarios  should  prescribe 
methods  of  using  the  device  for  individualized  or  peer  group 
training. 

Although  the  instructors  were  highly  )cnowledgeable  about 
most  of  the  aircraft  systems  which  they  taught,  and  were 
competent  in  the  operation  and  maintenance  of  these  systems, 
the  present  observations  provide  evidence  that  the  style  in 
which  they  delivered  the  instruction  may  have  lessened  overall 
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training  effectiveness.  The  instructors  often  presented  the 
material  in  a stereotypic  and  rushed  fashion,  and  they 
appeared  to  be  uneasy  in  presenting  a few  of  the  instructional 
unit.  These  effects  may  be  explained  partly  by  the  immense 
volume  of  information  which  had  to  be  presented,  but  this 
explanation  still  begs  the  question  of  poor  organization, 
as  well  as  the  general  perception  by  this  author  that  the 
instructors  probably  did  not  like  being  the  focus  of,  or 
having  responsibility  for,  much  of  the  instructional  delivery 
(at  least  under  the  present  conditions) . Organization  could 
be  improved  by  removing  some  of  the  information  (especially 
that  information  about  the  operation  and  maintenance  of 
system  components),  and  emphasizing  instead  (a)  the  normal 
and  emergency  display  data  which  pilots  or  NFOs  should  monitor 
routinely  during  ground  inspections,  pre-flight  checks,  and 
in-flight  operations,  and  (b)  the  prescribed  (NATOPS)  control 
operations  associated  with  these  displays.  (For  example, 
the  operation  of  inverters,  including  input  and  output 
voltages  and  currents,  or  a detailed  description  of  power 
plant  structure  and  operations,  such  as  the  number,  purpose 
and  operation  of  inner  guide  vanes  and  fuel  nozzles,  should  be 
eliminated  or  subsumed  under  more  advanced  training.) 

Training  delivery  support,  in  the  form  of  complete  training 
scenarios  which  integrate  the  ECII-LPs  with  other  training 
aids  to  meet  specific  behavioral  objectives,  should  also 
improve  instructional  organization  and  effectiveness.  Many 
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Qf  these  modifications  (scenario  planning,  reduction  of 
inappropriate  training  content,  development  of  better  training 
aids)  could  be  accomplished  by  the  instructors  if  they  could 
be  relieved  of  routine  and  repetitive  lecture  duties. 

The  above  observations  and  findings  indicate  that  both 
FAM  courses  would  probably  be  more  effective  if  the  courses 
were  managed  under  IMI  (Instructor  Managed  Instruction) 
procedures  than  if  limited  instructor  resources  continue  to 
be  used  primarily  for  lectures.  (The  small  student  load 
and  the  small  number  of  instructional  units  probably  does 
not  warrant  implementation  of  CMI — Computer  Managed  Instruction-- 
procedures  unless  some  cost-effective  interface  could  be  made 
with  a large  CMI  system  such  as  that  operated  by  the 
CNTECHTRA.>  IMI  would  not  only  provide  for  better  use  of 
training  aids  and  instructor  resources,  and  improve  motivation 
among  the  instructors,  but  the  resulting  standardization  would 
reduce  differences  in  student  performance  that  result  from 
the  highly  variable  lecture  and  delivery  styles  of  the  instruc- 
tors (including  instructional  organization,  the  use  of 
extraneous  material — i.e.,  "sea  stories"  and  jokes — and  the 
performance  differences  which  are  related  to  factors  such 
as  instructor  attitudes  and  voice  commi  ” ication — especially 
voice  projection  and  dialect) . 

6.2  Student  Findings.  As  mentioned  previously,  the 
aptitudes  and  aviation  experience  of  the  students,  especially 
those  students  enrolled  in  the  pilot  FAM  course,  was  highly 
mixed  and  diverse.  Even  among  the  more  homogenous  NFO  students, 
observations  indicated  that  the  aptitudes  required  to  learn 
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or  memorize  the  volume  of  information  which  was  presented 
varied  substantially.  This  author  noted  that  perhaps  as 
many  as  one-fourth  of  the  students  in  the  NFO  FAM  course 
were  being  assisted  ("carried")  by  other  students  to  some 
extent,  either  because  those  being  assisted  did  not  under- 
stand the  information  being  presented  or  because  they  had 
poor  study  skills  (or  some  combination  of  these  two  factors) . 
The  observation  was  also  made  that  specific  students  in  the 
NFO  course  were  primarily  responsible  for  taking  notes  and 
organizing  the  material  on  discrete  instructional  units 
(probably  as  a result  of  the  proficiency  which  they  demon- 
strated in  these  subject  matter  areas  in  previous  courses) . 
During  the  hourly  breaks  and  after  class,  the  students 
would  form  into  small  groups  of  three  to  six  students,  and 
one  or  two  members  of  these  small  groups  would  debrief  the 
other  students  on  the  material  just  covered.  Apparently,  the 
students  conducting  the  debriefing  were  recognized  by  the 
other  members  of  the  group  as  outstanding  in  these  subject 
matter  areas,  and  the  notes  which  they  took  and  the  subsequent 
debriefings  were  used  to  supplement  (not  substitute  for)  the 
information  gathered  by  the  other  students.  Inasmuch  as 
most  aviation  students  are  college  graduates,  this  procedure 
may  have  originated  from  the  practice  of  some  college  living 
groups  to  retain  the  notes  of  outstanding  students  in  specific 
college  courses  for  reference  and  study  purposes.  This  pro- 
cedure was  much  better  organized  in  the  NFO  FAM  course  than 
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in  the  pilot  FAM  course,  probably  because  the  NFO  class 
was  larger  (so  subject  matter  excellence  was  more  evenly 
distributed) , and  because  most  of  the  students  in  the  NFO 
FAM  course  (unlike  those  in  the  pilot  FAM  course)  were 
members  of  a single  training  §roup  or  class  which  has  been 
together  during  most  of  the  undergraduate  training  period 
(so  subject  matter  excellence  could  be  more  reliably  identified) . 
These  observations  appear  to  provide  evidence  for  the  follow- 
ing two  interrelated  effects:  (a)  the  formal  course  material 

or  structure,  at  least  in  the  NFO  FAM  course,  was  not  matched 
to  the  aptitudes  and  skills  of  individual  students  (either 
because  of  inadequate  training  support  and  organization,  or 
because  of  the  rapid  pacing) , and  that  (b)  these  training 
deficiencies  may  be  effectively  compensated  for  by  peer 
groups'  which  were  organized  much  earlier  in  the  undergraduate 
training  period.  Had  the  content  of  the  study  guides,  work- 
books, and  final  test  required  less  rote  memorization  of 
locations  and  functions  and  more  understanding  of  procedures 
and  the  relationship  among  systems  and  subsystems,  this  peer 
group  process  would  probably  have  been  less  effective  because 
the  material  would  have  been  more  difficult  to  organize  and 
integrate. 

The  training  effectiveness  of  the  ECII-LPs  is  difficult 
to  determine  under  these  comparison  conditions.  In  the  pilot 
FAM  course,  in  which  the  ECII-LPs  were  used,  the  students 
seemed  less  confused.  The  student  pilots  asked  fewer  questions 
than  the  student  NFOs,  especially  questions  which  duplicated 


22 


material  which  had  already  been  presented  by  the  instructor 
such  as  the  location  of  displays  and  controls  or  review  of 
basic  procedures.  The  questions  which  the  student  pilots  asked 
emphasized  variations  or  extensions  of  procedures  which  had 
been  described  previously  by  the  instructor.  In  addition, 
much  whispered  discussion  occurred  between  student  NFOs  while 
the  instructors  were  lecturing,  indicating  that  information 
was  being  missed  or  was  not  being  presented  in  a comprehensi- 
ble fashion.  The  student  NFOs  did  not  appear  to  comprehend 
the  functional  relationships  among  system  components  which 
were  presented  on  transparencies  or  other  training  aids  as 
well  as  the  student  pilots  understood  similar  information 
presented  on  the  ECII-LPs.  Although  student  use  of  the 
ECII-LPs  occurred  only  rarely,  the  occasions  on  which  the 
student  pilots  were  provided  with  an  opportunity  to  practice 
procedures  (such  as  hot  starts,  false  starts,  and  air  starts) 
appeared  to  be  highly  reinforcing  to  the  students  and  to  the 
learning  process.  Both  the  studdnts  who  were  practicing  on 
the  ECII-LPs  and  the  remaining  students  in  the  classroom 
participated  in  the  scenario,  and  they  responded  favorably 
to  this  opportunity  to  apply  and  integrate  the  information 
which  had  just  been  presented  by  the  instructor  (although  the 
scenario,  as  previously  noted,  may  not  have  been  well  planned) . 

The  differences  observed  above  between  student  pilots 
and  NFOs  (confusion  and  incomprehension  demonstrated  by 
student  NFOs  through  frequent  and  redundant  questions, 
whispered  discussions  during  lectures,  and  peer  group 
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debriefings)  could  be  attributed  to  several  factors,  including 
use  of  ECII-LPs,  the  larger  NFO  class  size,  the  shorter  NFO 
course,  and  the  lesser  operational  flight  experience  of  the 
student  NFOs.  Questions  about  the  location  and  operation 
of  displays  and  controls  by  student  NFOs  could  be  related  to 
(a)  inferior  presentation  of  this  information  by  the  avail- 
able transparencies  and  study  guides  compared  to  the  ECII-LPs 
(which  seemed  to  be  the  case  to  this  author) , or  (b)  to  the 
lesser  emphasis  placed  on  this  information  by  the  instructors 
in  NFO  classes.  As  mentioned  previously,  the  present 
observations  did  not  permit  a determination  of  the  extent  to 
which  each  of  these  factors  may  account  for  measures  of 
training  effectiveness.  Such  a determination  will  be  possible 
only  through  a formal,  summative  evaluation  such  as  that 
proposed  in  the  last  section  of  this  report. 

6.3  Discussion  of  Instructor  and  Student  Findings.  The 
above  observations  indicate,  and  this  author  strongly  recommends, 
that  a formal,  summative  evaluation  of  the  training  effective- 
ness of  the  ECII-LP  simulators  should  probably  not  be  under- 
ta)cen  until  minor  modifications  have  been  made  in  the  design 
and  support  of  the  ECII-LPs,  and  some  major  modifications  have 
been  instituted  in  the  management  and  delivery  of  the  existing 
pilot  and  NFO  FAM  courses.  Summative  evaluations  will  most 
liJcely  always  be  difficult  to  initiate  because  of  continual, 
minor  modifications  in  training  programs  and  delivery  systems, 
but  in  the  present  case,  in  which  both  major  and  minor  modifica- 
tions are  indicated,  conducting  such  an  evaluation  would  be 
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wasteful  of  personnel  and  funding  resources.  In  addition,  a 
suminative  evaluation  would  be  difficult  to  implement  in  the 
near  future  because  two  training  situations  do  not  currently 
exist  which  have  the  same  (or  similar)  students,  subject 
aircraft,  and  training  objectives,  and  which  differ  only  in  the 
availability  of  ECII-LPs.  (The  present  pilot  and  NFO  FAM 
courses  cannot  be  compared  validly  and  objectively  for  ECII-LP 
effectiveness  because  of  dissimilarities  in  students,  subject 
aircraft,  and  training  objectives.) 

The  major  problem  found  with  both  FAM  courses  is  that 
the  training  objectives  include  not  only  familiarization 
training,  but  have  apparently  been  expanded  to  include  basic 
cockpit  procedures  training  as  well.  Such  an  expansion  is 
probably  a normal  (although  unofficial  process  in  these  train- 
ing situations.  Strict  familiarization  training  should  encom- 
pass the  simplest  learning  levels  within  the  cognitive  domain 
of  training  developed  by  Bloom  (reference  5) . These  levels 
would  include  rote  memorization  of  the  identity  and  location  of 
displays  and  controls,  and  recognition  or  recall  of  the  basic 
information  which  is  conveyed  in  the  displays  or  the  primary 
operation  which  is  performed  by  activating  each  control. 
Training  problems  begin  to  arise,  however,  if  the  objectives 
also  include  an  understanding  of  the  systems  which  underlie 
displays  and  controls,  primarily  because  displays  and  controls 
are  usually  associated  with  several  systems  which  interact  in 
complex  ways.  By  knowing  these  interactions,  the  pilot  (or 
NFO  who  may  be  assisting  the  pilot  in  monitoring  the  displays) 
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can  determine  which  systems  to  activate  or  deactivate  should 
emergency  conditions  be  displayed.  In  order  to  understand  sys- 
tems interaction,  the  training  objectives  may  include  knowledge 
of  the  critical  points  (at  the  component  level)  at  which  various 
systems  interface  (the  points  at  which  functional  relationships 
between  systems  or  subsystems  are  established) . This  under- 
standing may  be  necessary  because  guages  or  sensors  are  often 
located  at  these  points  and  these  guages  and  sensors  are  usually 
checked  for  normal  operation  during  the  ground  or  pre-flight 
inspections.  Also,  by  knowing  whether  the  interface  between 
systems  is  electrical,  mechanical,  or  hydraulic,  the  pilot  or 
NFO  can  determine  whether  or  not  auxiliary  systems  will  be 
operative  under  specific  emergency  conditions  involving 
electrical,  mechanical,  or  hydraulic  failures.  This  information 
will  enable  the  pilot  or  NFO  to  perform  the  most  appropriate 
emergency  procedures.  Often,  several  interfaces  may  exist 
within  a single  system  (especially  the  electrical  system)  in 
which  several  parallel  or  auxiliary  subsystems  can  be  activated 
or  deactivated  in  order  to  maintain  power  to  specific  equipment 
under  emergency  conditions.  An  understanding  of  these 
subsystem  interfaces  requires  learning  and  memorizing  of  the 
function  of  many  components.  Unfortunately,  this  under- 
standing often  involves  the  presentation  of  complete  systems 
and  subsystems,  and  the  subsequent  learning  and  memorization 
of  unnecessary  components  that  are  intermediary  between  inter- 
face points  and  that  are  not  critical  in  the  detection  of 
failure  in  a critical  system  or  subsystem.  On  several  occasions 
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in  the  present  courses  (especially  in  the  E and  HFO  instructional 
units) , the  instructors  described  many  of  the  components  which 
could  be  involved  in  faults,  but  which  would  result  in  the 
same  display  information.  A noteworthy  observation  is  that  the 
instructors  would  usually  have  to  refer  to  the  study  guides, 
transparencies,  or  challc  boards  instead  of  the  ECII-LPs  to 
present  this  information.  Apparently,  the  information 
presented  on  the  ECII-LPs  was  more  selective,  and  limited  to 
that  information  which  was  necessary  only  to  detect  faults 
and  take  appropriate  action. 

The  above  discussion,  as  well  as  the  observations  made 
previously,  demonstrate  that  the  term  "familiarization"  for 
these  courses  may  be  a misnomer.  Both  the  pilot  and  NFO  FAM 
courses  consisted  not  only  of  familiarization  training,  but 
basic  cockpit  procedures  training  as  well.  Official  recognition 
of  this  basic  cockpit  procedures  training  should  be  made,  both 
in  the  title  and  in  the  organization  of  the  course.  Inasmuch 
as  the  next  course  taken  by  these  students  is  CPT  (Cockpit 
Procedures  Training) , a TSA  (Training  Situation  Analysis) 
should  be  made  of  the  training  objectives  which  are  actually 
trained  in  the  current  FAM  and  CPT  courses,  and  these  objectives 
should  be  integrated  in  order  to  avoid  overlap  and  to  make 
the  most  effective  use  of  available  training  aids  (especially 
the  ECII-LPs) . This  TSA  should  determine  which  component  and 
subsystem  information  is  truly  enabling  in  the  development  of 
terminal  training  objectives  in  the  CPT  and  aircraft. 
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The  above  ob«5ervations  and  interviews  indicate  that  the 
nresent  FAM  courses  could  be  divided  into  at  least  three  phases, 
with  the  FCII-LPs  being  used  only  in  the  final  phase.  Phase  1 
would  be  strictly  familiarization  training,  involving  recall 
of  the  identity  and  location  of  displays  and  controls,  as  well 
as  recall  of  the  primary  purpose  and  operation  of  these  displays 
and  controls.  This  phase  should  expand  on  that  familiarization 
information  which  is  currently  offered,  and  present  this 
information  in  a better  organized  fashion,  free  of  functional 
system  descriptions  and  procedures  (which  would  be  presented 
in  the  next  two  phases) . This  information  could  be  effectively 
presented  in  well  prepared  study  guides,  transparencies,  and 
audio-visuals  (sound/slide) . Phase  2 would  consist  of  under- 
standing critical  system  and  subsystem  interfacing,  including 
functional  relationships  between  systems  and  subsystems  under 
normal  and  emergency  conditions.  This  training  could  also  be 
accomplished  with  the  above  training  aids.  (This  phase  would 
most  represent  the  FAM  courses  which  are  taught  presently,  but 
would  consist  only  of  that  information  which  is  necessary  to 
understand  the  functional  relationships  among  systems  and 
subsystems  and  which  is  associated  with  activation  and  deactiva- 
tion of  displays  and  controls.)  The  final  phase  would  be  a 
practice  or  applications  phase  in  which  the  ECII-LPs  would  be 
used  to  present  normal  and  emergency  scenarios  to  the  students. 
Students  would  practice  ground  inspections,  pre-flight  checks, 
and  normal  taxi,  take-off,  and  in-flight  procedures,  as  well  as 
emergency  procedures  (including  hot  starts,  air  starts,  and 
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activation  of  OLS  and  CE  equipment)  during  this  phase,  (CE 
procedures  training  would  be  contingent  on  procuring  an  ECII-LP 
simulator  for  this  purpose.)  These  three  phases,  arranged  in 
this  order  and  emphasizing  the  proposed  content  and  objectives, 
would  offer  familiarization  and  procedures  training  in  a pro- 
gressive, systemmatic  fashion,  and  would  therefore  conform 
closely  to  the  first  three  cognitive  domain  levels--)cnowledge , 
comprehension,  and  application — established  by  Bloom  (reference 
5)  . 

As  mentioned  previously,  phases  I and  II  could  most  easily 
and  effectively  be  managed  using  IMI  procedures,  in  which 
instruction  would  be  individualized  and  an  instructor  would 
score  pre-  and  post-tests,  determine  the  next  unit  of  instruc- 
tion, and  discuss  instructional  problems  with  each  student. 

The  IMI  procedure  has  been  found  to  be  cost-  and  training- 
effective  for  aviation  maintenance  students  (reference  6), 
and,  as  described  above,  the  IMI  procedure  appears  to  be 
suitable  for  the  present  FAM  training  courses  as  well. 

Phase  3 would  consist  of  the  applications  and  integration 
training  levels  developed  by  Bloom.  In  this  phase,  information 
from  the  previous  two  phases  would  be  integrated  and  used  to 
practice  and  solve  procedural  problems  in  the  ECII-LPs.  Some 
psychomotor  procedural  s)cills  would  also  be  trained,  although 
most  of  the  objectives  in  the  psychomotor  domain  are  probably 
better  trained  in  the  CPT  or  aircraft.  This  author  would 
prefer  that  the  ECII-LPs  be  used  in  an  individualized/IMI  mode 
for  phase  3.  This  mode  would  be  most  effective  in  correcting  v 
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procedural  problems  which  are  unique  to  each  student,  and  would 
also  take  maximum  advantage  of  the  interactive  (reinforcement) 
features  of  the  ECII-LPs.  Such  use,  however,  should  not  be 
formally  adopted  until  an  evaluation  has  demonstrated  that  use 
of  the  ECII-LPs  in  this  mode  reduces  the  number  of  hours 
required  on  CPT  simulators  or  in  the  aircraft,  or  that  the 
students  entering  these  latter  two  training  situations  enter 
at  a more  proficient  performance  level  than  students  who  have 
used  another  ECII-LP  training  mode.  The  next  most  cost-  and 
training-effective  option  probably  would  be  to  have  small 
groups  of  students  (not  more  than  six)  practice  normal  and 
emergency  procedures  on  the  ECII-LPs.  This  option  would  take 
advantage  of  the  peer  groups  which  appear  to  have  already  been 
established  in  some  of  these  courses,  and  would  permit  limited 
reinforcement  of  cognitive  skills  learned  in  the  previous  two 
phases.  Unfortunately,  students  will  most  likely  volunteer  to 
practice  those  procedures  which  they  understand  best,  and  may 
get  little  practice  on  those  procedures  which  they  know  least 
well.  (This  effect  may  be  minimized,  however,  by  having  each 
student  outline  the  steos  of  a procedure  in  writing  first,  and 
then  select  several  different  outlines  from  among  the  group  to 
demonstrate  on  the  ECII-LP.)  The  least  preferred  option  is  to 
use  the  ECII-LPs  in  the  current  lecture/demonstration  mode. 
Although  this  mode  will  probably  be  of  some  training  benefit, 
the  unique  training  performance  deficiencies  of  many  students 
will  have  to  await  identification  and  correction  at  some  later 
stage  in  training  (in  the  CPT  or  aircraft--which  is  not  an 
effective  use  of  the  CPT  or  aircraft) . 
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Whichever  of  the  above  inodes  is  used,  the  requirement  will 
still  exist  to  provide  the  ECII-LP  with  improved  maintenance 
and  training  scenario  support.  As  mentioned  previously,  more 
should  be  done  to  supply  the  NAMTRADETs  with  ECII-LP  schematics 
and  parts,  as  well  as  complete  training  (procedures)  scenarios 
and  associated  documentation.  The  development  of  programmed 
scenarios  and  documentation  is  probably  beyond  the  resources 
available  in  the  NAMTRADETs  and  will  have  to  be  procured  from 
other  sources  with  planning  assistance  from  the  NAMTRADETs. 

In  addition,  the  ECII-LP  visuals  should  be  modified  as  men- 
tioned previously-larger  rear  projection  screens  and  use  of 
those  visuals  which  contain  only  critical  procedures  information 
or  interactive  information  about  the  correctness  of  performance 
(reinforcement) . The  IMI  procedures  proposed  for  phases  1 and 
2,  however,  should  free  the  instructors  to  improve  the  organiza- 
tion and  delivery  of  training  materials  associated  with  these 
phases.  Although  NAMTRADET  personnel  most  li)cely  could 
manage  effectively  the  three  training  phases,  the  procedural 
context  of  phase  3 and  the  transitional  relationship  which  this 
phase  (and  the  ECII-LPs)  has  with  the  CPT  and  aircraft  may 
require  that  this  phase  be  managed  by  qualified  instructor 
pilots/NFOs  at  the  squadron  level. 

6.4.  Summative  Evaluation  Design.  The  above  discussion  and 
recommendations  do  not  advocate  that  the  number  of  instructors 
or  the  number  of  course  hours  necessarily  be  reduced.  In 
some  instances,  such  as  phases  1 and  2,  substantial  expansion 
of  the  present  course  can  be  expected,  at  least  in  the  initial 
training  evolutions.  The  above  observations  ac)cnowledge  that 
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these  courses  are  currently  understaffed.  The  purpose  of 
instituting  IMI  procedures,  therefore,  would  be  to  make 
instructors  available  to  better  organize  training  managen^nt 
and  delivery  to  meet  the  behavioral  objectives  more  effectively. 
By  reducing  or  eliminating  lecture  duties  and  having  instructors 
assume  responsibility  for  the  construction  of  valid  pre-  and 
post-test  performance  measures,  development  of  training  aids, 
and  documentation  and  revision  of  the  training  management 
process,  some  reduction  in  initial  course  hours  may  occur  after 
several  training  evolutions.  The  validity  of  these  recommended 
training  modifications,  however,  will  be  whether  or  not  the 
students  are  better  trained  for  the  CPT  and  aircraft,  demon- 
strated by  improved  entrance  level  performance  on  the  CPT  and 
aircraft,  or  by  a reduction  in  CPT  and  aircraft  hours.  (Another 
effectiveness  measure  would  be  the  training  of  more  procedures-- 
especially  emergency  procedures — within  the  number  of  hours 
currently  allotted  to  the  CPT  and  aircraft.)  A summative 
evaluation,  therefore,  should  measure  the  effectiveness  of  the 
various  ECII-LP  training  modes  recommended  in  phase  3 (individ- 
ualized, peer  group,  and  lecture/demonstration)  through  entrance 
level  performance  measurements  and  instructor  ratings,  average 
total  number  of  instructional  hours  (or  hours/unit) , and  average 
total  number  of  emergency  procedures  practiced  in  the  CPT  or 
aircraft.  These  criteria  of  effectiveness  should  be  used 
regardless  of  whether  or  not  the  above  training  modifications 
are  implemented.  Using  the  current  written  final  test  would 
not  be  valid  because  this  test  does  not  contain  a sufficient 
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number  of  items  to  measure  the  procedures  training  which  is 
actually  being  accomplished.  The  final  test  scores  also  do 
not  represent  a random  unbiased  sample  of  the  classroom  training 
which  occurs  because  the  instructors  prompt  the  students  about 
the  final  test  items.  In  addition,  the  peer  group  process 
most  likely  compensates  for  many  of  the  classroom  training 
deficiencies  and  differences  that  might  be  demonstrated  in 
final  test  scores.  The  initial  ground  inspection  ("walk 
around")  and  pre-flight  scores  which  are  obtained  by  the  student 
from  the  instructor  pilot  on  arrival  at  the  squadron  probably 
are  more  valid  effectiveness  criteria  than  written  final  test 
scores  (and  would  be  the  only  valid  criteria  if  the  FAM 
courses  involved  only  familiarization  training).  These  criteria, 
however,  would  fail  to  measure  many  of  the  procedural  behaviors 
that  are  learned  in  the  present  courses  (and  these  procedural 
behaviors  are  emphasized  even  more  in  the  proposed  modifica- 
tions) . These  criteria  should  be  included,  however,  as  measures 
of  the  training  effectiveness  of  phase  1.  Only  by  measuring 
the  effectiveness  of  each  of  the  three  ECII-LP  training  modes 
on  CPT  and  aircraft  procedures  training  will  the  effectiveness 
of  the  ECII-LPs  be  validly  tested  and  recognized. 

If  the  above  course  modifications  are  implemented,  and 
the  current  ECII-LPs  are  used  in  an  IMI  mode,  then  procurement 
of  the  ECU  (desk  console  model)  would  be  most  appropriate  in 
the  future.  Although  the  current  ECII-LPs  probably  are  as 
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training  effective  as  the  ECUs,  the  ECUs  would  be  more  cost- 
effective  and  convenient  for  individualized  training.  In 
addition,  the  development  of  the  EC-IIIs  described  in  section  4.0 
should  be  followed  closely  because  these  simulators  promise 
to  be  highly  effective  for  training  many  of  the  complex  psycho- 
motor skills  currently  being  trained  on  CPTs.  The  EC-IIIs,  in 
combination  with  the  ECUs  (and  using  the  scenario  software 
development  and  programming  which  should  be  made  available  to 
the  current  ECUs)  , has  the  probability  of  being  a highly 
training-  and  cost-effective  mix. 

7.0  CONCLUSIONS . The  conclusions  listed  below  have  been  drawn 
from  the  above  results: 

. 1 Comparative  bbservations  between  the  NFO  FAM  course  which 
used  inexpensive  training  aids  (transparencies,  chalk  boards, 
and  study  guides)  and  the  pilot  FAM  course  which  used  ECII-LP 
simulators  indicate  that  the  two  courses  did  not  differ  sub- 
stantially in  training  effectiveness  (as  inferred  from  student 
questioning,  interviews,  and  reviews  of  written  post-test  scores). 
This  conclusion  is  highly  tentative,  however,  because  of  differ- 
ences in  aircraft,  student  aptitudes  and  skills,  peer  group 
processes,  and  training  objectives  between  the  two  courses. 

The  absence  of  objective  training  performance  criteria  for  these 
courses  also  hinders  any  conclusion  about  comparative  training 
effectiveness.  "v 

7.2  The  two  FAM  courses  (NFO  and  pilot)  consist  not  only  of 
familiarization  training,  but  basic  cockpit  procedures  training 
as  well. 
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7.3  Although  the  instructors  appear  to  be  highly  conpetent 

in  the  maintenance  of  special  aircraft  systems,  they  vary  sub- 
stantially in  instructional  skills,  especially  lecture  style 
and  management  of  course  organization  and  content. 

7.4  The  aotitudes,  skills,  and  aviation  backgrounds  of  the 
students  also  varied  substantially,  and  these  factors  appear  to 
be  related  to  the  training  effectiveness  of  the  two  FAM  courses. 
In  the  larger  of  the  two  courses  (NFO  FAM) , an  informal  peer 
group  training  process  seemed  to  have  developed  in  order  to 
provide  the  poorer  students  with  the  additional  training 
necessary  to  pass  the  course. 

7.5  The  instructors  appear  to  favor  the  ECII-LPs  over  other 
training  aids  and  devices,  primarily  because  the  ECII-LPs  were 
easy  to  move,  fairly  compact,  accessible,  and  organized  the 
subject  matter  at  a central  location.  The  graphics,  especially 
those  on  the  panels,  were  accurate  and  well  organized.  The 
larger  class,  however,  had  some  difficulty  viewing  the  panel 
graphics  from  the  back  of  the  room.  The  basence  of  structural 
fidelity  seemed  to  be  more  than  compensated  by  the  functional 
programming  features  of  the  simulator,  as  well  as  the  overall 
ease  of  use. 

7.6  Several  human  factors  deficiencies  were  found  in  the  simula- 
tors. First,  the  rear  projection  visual  system  was  totally 
inadequate.  The  slides  (except  those  containing  only  words) 
were  difficult  to  see  from  almost  any  distance,  and  the  visual 
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system  was  not  synchronized  with  the  rest  of  the  training 
scenario.  These  problems  seemed  to  reduce  instructor  accep- 
tance and  use  of  the  programmed  training  scenarios.  The  second 
deficiency  was  that  the  panels  were  too  heavy  and  cumbersome  to 
be  replaced  by  a single  instructor. 

7.7  Instructors  were  not  prepared  to  use  the  ECII-LPs.  They 
had  difficulty  completing  a programmed  training  scenario  (even 
without  mechanical  or  programming  problems) , and  they  did  not 
integrate  these  training  scenarios  adequately  with  the  other 
course  materials.  Most  students  did  not  have  an  opportunity 
to  operate  the  ECII-LPs,  although  they  seemed  to  favor  the 
problem-solving  and  reinforcing  features  of  the  simulator. 

7.8  The  ECII-LPs  have  a high  level  of  engineering  reliability. 
Engineering  faults,  however,  are  difficult  to  correct  because 
parts  are  not  readily  available  and  NAMTRADET  personnel  have 
not  been  trained  formally  to  maintain  the  simulators. 

8.0  RECOMMENDATIONS . The  following  recommendations  are  made 
in  association  with  the  above  conclusions: 

8.1  Although  the  ECII-LPs,  as  presently  used,  do  not  appear  to 
improve  training  effectiveness,  the  simulators  could  be  made 
more  training  effective  if  used  properly.  Effectiveness  is 
dependent  on  official  recognition  that  the  FAM  courses  should 
include  training  objectives  in  basic  cockpit  procedures,  and 
providing  the  instructors  with  complete  training  scenarios 
which  are  associated  with  specific  behavioral  objectives.  The 
costs  involved  in  expanding  the  FAM  courses  to  provide  this 
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training  should  be  more  than  compensated  by  better  aircraft 
performance  and  more  efficient  use  of  CPTs.  If  the  courses 
are  not  modified  to  provide  this  training,  then  course  content 
should  be  reduced  to  familiarization  training  only.  Strict 
FAM  training  should  be  limited  to  recall  of  the  identity  and 
location  of  displays  and  controls,  as  well  as  recall  of  the 
primary  purpose  and  operation  of  these  displays  and  controls. 

FAM  training  should  not  involve  cognitive  integration  of 
display  and  control  information  in  a problem-solving  situation. 

8.2  The  training  problems  associated  with  variations  in 
student  aptitudes,  skills,  and  experience,  as  well  as  differ- 
ences in  instructor  lecture  style  and  course  management,  could 
be  corrected  by  implementing  IMI  (Instructor  Managed  Instruction) 
procedures.  Under  IMI  (and  assuming  that  the  courses  would 

be  expanded  to  include  basic  cockpit  procedures  training)  the 
students  should  be  provided  with  substantially  more  opnortunity 
to  reinforce  knowledge  skills  by  practicing  programmed  training 
scenarios  on  the  ECII-LPs,  either  alone  or  in  small  groups. 

Use  of  the  ECUs  (vice  ECII-LPs)  may  be  warranted  in  these 
training  situations.  The  prospective  ECIIIs  also  appear  to  be 
appropriate  for  this  tyoe  of  training. 

8.3  Engineering  solutions  should  be  found  for  the  problems  of 
panel  replacement  and  the  inadequate  rear  projection  visual 
system  (including  improved  visuals  and  correct  synchronization 
with  the  programming  system) . 
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8.4  The  instructors  should  be  provided  with  completely 
documented  training  scenarios,  as  well  as  lesson  plans  which 
integrate  the  ECII-LPs  with  other  training  aids  and  devices. 

The  instructional  objectives  which  are  to  be  met  by  the  ECII-LPs 
should  also  be  specified. 

8.5  Instructors  should  be  provided  with  0-level  maintenance 
training  for  the  simulators,  as  well  as  a stock  of  necessary 
parts . 

8.6  Until  the  above  modifications  are  made  in  the  present 
courses  and  in  the  use  of  the  ECII-LPs,  a summative  evaluation 
of  the  ECII-LPs  objective  measures  of  training  performance 

(in  addition  to  final  test  scores)  should  be  developed. 
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APPENDIX  A 


Normal  and  Exploded  Views  of  the 
ECU  and  ECII-LP  Simulators 


40 


Normal  Viow  of  I-.CII 


' n*  *"| 

Li*  I 1 

rr*  4>  ^;i 

^ \ 

PROJECTOR  SCREEN 


ELEVATOR  ASSEMBLY 


TRAINING  PANEL 


METER  SCALE 


SLIDE  DISK 


PROJECTOR 


CONTROL 

CONSOLE 


PROGRAM  / 
TAPE  CASSETTE  ^ 


AUDIBLE 

ALARM 

meterX 


COMPUTER 


POWER  SUPPLY 


ELEVATOR 
DRIVE  MOTOR 


ACCESS  PANEL  (4) 


CABINET  ASSEMBLY 


Exploded  view  of 


EC  11-l.P  Trainer  Console 


DISTRIBUTION  LIST  12-75 


Assistant  Secretary  of  the  Navy  (Manpower  & Reserve  Affairs)  (2) 
Chief  of  Naval  Operations: 

(OP-39) 

(OP-59) 

(OP-099)  (2) 

(OP-964) 

(OP-987E) 

Vice  Chief  of  Naval  Operations 
Chief  of  Naval  Personnel: 

(Pers-Od)  (Pers-212) 

(Pers-5)  (Pers-52) 

(Pers-6)  (Pers-55) 

(Pers-2x)  (Pers-61) 

Chief  of  Naval  Research: 

(Code  450)  (4) 

(Code  458)  (2) 

Chief  of  Naval  education  and  Training: 

(CNET  N-2) 

(CNET  N-4) 

(CNET  N-5) 

(CNET  N-51) 

Chief  of  Naval  Material  (NAVMAT  03T2) 

Commander,  Naval  Recruiting  Command 
Chief  of  Naval  Air  Training  (2) 

Chief  of  Naval  Technical  Training  (2) 


Office  of  Assistant  Secretary  of  Defense  (M&RA) 

Commander  Training  Command,  U.  S.  Pacific  Fleet 
Commander  Training  Command,  U.  S.  Atlantic  Fleet 
Commander,  Naval  Training  Center,  Great  Lalces  (2) 

Commander,  Naval  Electronics  Laboratory  Center  (1) 

Commanding  Officer,  Manpower  & Material  Analysis  Center,  Pacific 
Commanding  Officer,  Naval  Health  Research  Center,  San  Diego 
Commanding  Officer,  Naval  Aerospace  Medical  Institude  (1) 
Commanding  Officer,  Naval  Education  and  Training  Program 
Development  Center  (2) 

Commanding  Officer,  Naval  Submarine  Medical  Center  (1) 

Commanding  Officer,  Naval  Medical  Research  Institute 
Naval  Personnel  Research  & Development  Center,  San  Diego 
Commanding  Officer,  Service  School  Command,  Naval  Training 
Center,  Great  Lakes 

Commanding  Officer,  Service  School  Command,  Naval  Training 
Center,  Orlando 

Commanding  Officer,  Service  School  Command,  Naval  Training 
Center,  San  Diego 


45 


DISTRIBUTION  LIST  12-75  (Continued) 


Commanding  Officer,  Naval  Training  Equipment  Center 
Commanding  Officer,  Naval  Education  and  Training  Support 
Center,  Pacific 

Commanding  Officer,  Naval  Education  and  Training  Support 
Center,  Atlantic 

Director,  Naval  Instructional  Technology  Development  Center 
Director,  Training  Analysis  and  Evaluation  Group 
Center  for  Naval  Analyses 

U.  S.  Army  Enlisted  Evaluation  Center  (1) 

Human  Resources  Development  Division,  U.  S.  Army  Personnel 
4<  Administration  Combat  Developments  Activity,  Fort 
Benjamin  Harrison 

Army  Research  Institute  for  Behavioral  & Social  Sciences 
Personnel  Research  Division,  Air  Force  Human  Resources 
Laboratory  (AFSC) , Lackland  Air  Force  Base  (1) 
Occupational  Research  Division,  Air  Force  Human  Resources 
Laboratory  (AFSC) , Lackland  Air  Force  Base 
Headquarters,  U.  S.  Marine  Corps  (Code  MPI) 

Commandant,  U.  S.  Coast  Guard  (Code  B-5) 

Superintendent,  U.  S.  Naval  Academy 
Superintendent,  U.  S.  Air  Force  Academy 
Superintendent,  Naval  Postgraduate  School 
Superintendent,  U.  S.  Coast  Guard  Academy 
National  Science  Foundation 

Director,  Defense  Documentation  Center  (12) 


UNCLASSIFIED 


security  classification  of  this  page  CXTiFn  Dmi*  Bnitrmd) 


REPORT  DOCUMENTATION  PAGE 


' REPORT  Number 

CNETS  RBPOirP-2-76 


Q>pp  READ  INSTRUCTIONS 

bEFORE  COMPLETING  FORM 

2 GOVT  ACCESSION  NO.  3 RECIPIENT'S  CATALOG  NUMBER 


4 title  land  Subllll.; 5 TYPE  OF  REPORT  & PERIOD  COVEREO 

/observations  on  the  Use  and  Evaluation 

of  ECII-LP  Simulators  for  Aviation  ^ 

fp-  ^ 1 « PERFORMING  ORG.  REPORT  NUMBER 

irainxngji  ^ / 


7 AuTmOR^'«> 

fpobert  J./siersner 


• . CONTRACT  OR  grant  NUM8ER<'«; 


9 PERFORMING  ORGANIZATION  NAME  AND  ADDRESS  10  PROGRAM  El  EMEnT  PROJECT.  TA«F 

AREA  a WORK  UNIT  NUMBERS 

Chief  of  Naval  Education  and  Training 
Support,  Pensacola,  FL  32509 

" CONTROLLING  OFFICE  NAME  and  ADDRESS  >2  IREPORTDaYe 


" CONTROLLING  OFFICE  NAME  and  ADDRESS  >2  IREPORTDaYe 

October  13/6 

J.1  -u 

14  monitoring  AGENCY  NAME  » AODRESSf// dl/IerwK  (roni  Con, roU/ni  0«(c»>  '5  SECURITY  CLAjS  fol  ,M«  r.porfj 

' UNCI.ASSIFIED 

IS#'  OECL  aIsi  FI c ation  ^oowngradin o 
SCHEDULE 

li  distribution  STATEMENT  fol  Ihit  Rmporl) 

Approved  for  public  release;  distribution  unlimited. 


17  Distribution  statement  (of  thm  mbotrmct  ontorod  In  Block  20,  If  dlfforont  from  Rmport) 


18  SUPPU  EMEN  T A «Y  not  E5 


19  KEY  W'jROS  ^ConfJnu*  on  rovorao  oldo  If  nocooomry  and  Idmntlfy  by  block  nutnbor) 


Training  Effectiveness 
Simulators 
Student  Pilot 
Instructor  Pilot 


Training  Objectives 
Instructor  Managed 
Instruction 
Cockpit  Procedures 
Trainers 


ECU 
ECII-LP 
Fidelity 
Desk  Console 
Mini -Compute  r 


20  abstract  fConlfnu«  on  r«v«r»»  If  n«c»>«*ry  and  Idantlfy  by  biocfc  numbar) 


An  evaluation  of  the  training  effectiveness  of  ECII-LP  simulators 
was  undertaken  using  the  participant-observer  method.  Compari- 
sons were  made  between  two  aviation  courses — pilot  FAM  which 
used  the  ECII-LPs  to  familiarize  student  pilots  and  instructor 
pilots  with  the  T-2C  aircraft,  and  NFO  FAM  which  used  trans- 
parencies, study  guides,  MTUs,  and  chalk  boards  to  present 
familiarization  training, to  NFO  students  who  were  transitioning 


DD  W3A  edition  OF  I NOV  89  |$  OBSOLETE 

S/N  O102*0M-«80l  f 


UNCLASSIFIED  /j' 

JECURITY  cl AttIFICATlON  OF  TmiVTaOE  »nl«,AVI 


UNCLASSIFIED 


uwiiv  CI*SSIF1C»I10N  OF  This  PAoerH'h.o  Dmim  Bnitnd) 


to  the  TA-4  aircraft.  Interviews,  student  questions,  and  a I 

review  of  final  test  scores  showed  that  the  two  courses  were  j 

probably  equally  effective  in  meeting  the  training  objectives  ! 
if  allowances  are  made  for  differences  in  student  aptitudes 
and  skills,  previous  aviation  experience,  and  length  of  the  i 

training  period.  The  observations  found,  however,  that  the 
courses  also  provided  training  in  basic  cockpit  procedures  j 

although  the  final  tests  did  not  emphasize  these  objectives 
nor  were  the  ECII-LPs  being  used  effectively  for  this  type  of 
training.  Although  the  ECII-LPs  were  highly  mobile  and  presented, 
training  information  in  a well  organized  fashion,  the  rear 
projection  visual  system  of  the  simulator  was  found  to  be 
inadequate,  and  pan  1 replacement  was  difficult.  Instructors 
were  not  provided  w'.th  sufficient  training  or  support  in  the  j 

use  and  maintenance  of  the  simulators,  nor  were  necessary  j 

replacement  parts  readily  available.  The  reinforcing  and  I 

problemsolving  features  of  the  simulators  were  seldom  used,  : 

nor  were  students  allowed  to  operate  the  simulators  alone  or 
in  small  groups . ,,>-Recommendations  are  made 'that  {/Sj  the  basic- 
cockpit  procedures  training  objectives  of  the  FAM  courses  be 
recognized  officially,  (B)  completely  documented  training 
scenarios  which  relate  simulator  use  to  specific  behavioral 
objectives  be  developed;  better  maint^ance  support--  - 

O- level  training  and  parts— -be  provided,  (?)  improvements  j 

be  made  in  ^e  reac  projection  visual  system  and  panel  replace- 
ment, and  (^)  flnstructor  Managed  Instruction)  procedures 

be  adopted  in  order  to  avoid  problems  associated  with  differ- 
ences in  student  aptitudes,  skills,  and  experience,  as  well  as 
variations  in  instructor  delivery  and  organization.  Implemen- 
tation of  these  recommendations  should  result  in  better  use  j 

of  CPTs  (Cockpit  Procedures  Trainers)  and  improved  aircraft  ! 

performance.  A summative  evaluation  of  the  ECII-LPs  should 
be  delayed  until  major  course  modifications  have  been  made  » 
and  objective  performance  measures  of  training  effectiveness 
have  been  developed. 


UNCLASSIFIED 

•«CU«tTY  CLAttIFICATION  OF  THIl  PAOe(T»A»"  Omim  Bnltnd) 

w ^ 


